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(57) Abstract 

A medical instrument that comprises: a first-stage optic responsive to a tissue surface of a patient; a spectral separator optically 
responsive to the first stage optic and having a control input; an imaging sensor optically responsive to the spectral separator and having an 
image data output; and a diagnostic processor having an image acquisition interface with an input responsive to the imaging sensor and a 
filter control interface having a control output provided to the control input of the spectral separator. 
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MULTISPECTRAL/HYPERSPECTRAL MEDICAL INSTRUMENT 

Field of the Invention 

The invention relates to a surgical and diagnostic instrument for performing real-time 
general-purpose imaging during surgery, clinical procedures, or other medical evaluations. 

Background of the Invention 

Spectroscopic imaging devices which employ Acousto-Optic Tunable Filters (AOTF), 
Liquid Crystal Tunable Filters (LCTF), or dispersive gratings are known. Such devices have 
been used for microscopy and remote sensing. 

Summary of the Invention 

Generally, the invention features a medical instrument that includes an optic responsive 
to a surface of tissue of a patient, a spectral separator optically responsive to the optic, and an 
imaging sensor optically responsive to the spectral separator. The instrument also includes a 
diagnostic processor having an image acquisition interface responsive to the imaging sensor and 
a filter control interface to which the spectral separator is responsive. 

The spectral separator can be a tunable filter, such as a liquid crystal tunable filter, and 
the imaging sensor can be a two-dimensional imaging array, such as a charge coupled device. 
The optic can include a macro lens, an adjustable lens, or a probe that includes an imaging fiber 
optic cable, and a stand can be connected relative to the optic to position the optic relative to the 
patient. The control interface can be operable to adjust the filter at least twenty times to acquire 
hyperspectral data for redisplay in real time. The medical instrument can perform diagnostic 
processing for images acquired exclusively under visible light. 

The diagnostic processor can also include a general-purpose processing module and 
diagnostic protocol modules, which can each include filter transfer functions and an image 
processing protocol. The general-purpose processing module can be operative to instruct the 
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filter to successively apply the filter transfer functions to light collected from the patient, to 
acquire from the imaging sensor a number of images of the collected light each obtained after 
one of the filter transfer functions is applied, and to process the acquired images according to the 
image processing protocol to obtain a processed display image. The general-purpose processor 
can be a real-time processor operative to generate a processed display image within a time period 
on the order of the persistence of human vision. It may also be operative to acquire some images 
more slowly depending on the number of wavelengths and complexity of diagnostic processing 
protocol. The sensor and filter can be operative in the visible, infra-red, and UV regions. 

Instruments according to the invention are advantageous in that they can permit a surgeon 
or a physician to diagnose a medical condition or develop a surgical strategy based on real-time 
images obtained during surgery or in the course of performing clinical procedures or other 
medical evaluations. The physician may therefore be able to obtain significantly more 
information about a patient's condition than he might otherwise have been able to assemble by 
presenting an interactive interface. This additional information may permit a given surgical 
procedure to be carried out more precisely and may lead to more successful surgical results. It 
may also enhance the precision and results of other types of medical evaluations and procedures. 

The general-purpose nature of the instrument can also help the surgeon develop 
significant amounts of medical information in time-critical surgical situations. For example, a 
patient may undergo relatively straight-forward surgery during which the surgeon may discover a 
tumor or another internal condition. With an instrument according to the invention, the 
physician can spend a small amount of additional time with the patient under anesthesia and 
determine the nature and extent of the tumor. This can be particularly beneficial during major 
surgery, where extending surgery duration poses a potential morbidity and mortality risk. 
Because the procedure is rapid and noninvasive, the patient is exposed to little additional risk. 
The benefit of immediate diagnosis and evaluation is significant. 

An instrument according to the invention may also be able to provide a wide variety of 
diagnostic capabilities, allowing a physician to enhance the capabilities of his or her practice 
substantially in a variety of different realms, without investing in a number of instruments. The 
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physician can then enhance or update the instrument by the adding software modules that are 
specifically targeted towards certain conditions of particular tissues, subsystems, or disease 
states. This can allow a single base instrument to be configured for a variety of different types of 
practices, and priced according to the type of practice to be served by the instrument. For 
example, a general-purpose instrument to be used by a general surgeon could include a package 
of diagnostic protocols that would permit the diagnosis of a variety of conditions that a general 
surgeon might encounter, while a neurosurgeon's module might be added to allow a specialist to 
detect particular conditions within the brain. Electronic and optical upgrades may also be 
provided to update, specialize, or improve the performance of the instrument. Such upgrades can 
include processing modules, memory boards, lenses, and the like. 

Brief Description of the Drawings 

Fig. 1 is a perspective diagram of a macroscopic instrument according to the invention; 

Fig. 2 is a perspective diagram of a rigid or flexible probe-based instrument according to 
the invention; 

Fig. 3 is a block diagram of the instrument of Fig. 1; and 

Fig. 4 is a flowchart illustrating the operation of the system of Fig. 1. 

Detailed Description of an Illustrative Embodiment 

Referring to Fig. 1, an instrument according to the invention 10 includes an imaging 
module 12 mounted on a surgical stand 14. In this embodiment, the surgeon can direct the 
imaging portion 12 towards a patient 20 by manipulating a control 16 that adjusts the attitude of 
the imaging portion through a positioning mechanism 18. 

Referring to Fig. 2, an alternative embodiment of the invention 22 may include a probe 
such as a rigid or flexible endoscopic, thoracoscope, laproscopic, or angioscopic probe 24 
connected to an imaging station 30 via a fiber-optic cable 26. The surgeon can manipulate the 
probe within the patient in a minimally-invasive surgical procedure and derive images from a 
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portion of the patient and display these images on a display 28. A medical implement 32, such 
as a laser, can also be provided through the probe. For example, after diagnosing a particular 
condition, a physician can begin laser ablation therapy to remedy it. 

Referring to Fig. 3, a medical instrument according to the invention 34 may include an 
optical acquisition system 36 and a diagnostic processor 38. The acquisition system 36 includes 
a first-stage imaging optic 40, a Liquid Crystal Tunable Filter (LCTF) 42, a second-stage optic 
44, and an imaging element 46. The first-stage optic receives light collected from the patient and 
focuses it onto the surface of the LCTF. The first-stage optic can be a simple or compound 
macro lens in the case of a macroscopic instrument (Fig. 1). In a probe-based instrument (Fig. 
2), the first stage optic can include imaging optics within a probe such as a endoscopic, 
laproscopic, thoracoscopic, or angioscopic probe. The first stage lens can also be adjustable, 
allowing a physician to scan larger areas of tissue and then zoom into particular regions. 

The LCTF 42 is a programmable filter that filters out all but a wavelength region of 
interest from the light collected from the patient. The second-stage optic 44 receives the 
remaining light from the LCTF and focuses it onto the image sensor 46. The image sensor is 
preferably, although not necessarily, a two-dimensional array sensor, such as a charge-coupled 
device array (CCD), which delivers an image signal to the diagnostic processor 38. 

The diagnostic processor 38 includes an image acquisition interface 50, that has an input 
responsive to an output of the image sensor 46 and an output provided to a general-purpose 
operating module 54. The general-purpose operating module includes routines that perform 
image processing, and that operate and control the various parts of the system. It has a control 
output provided to a filter control interface 52, which in turn has an output provided to the LCTF 
42. The general-purpose operating module also interacts with a number of diagnostic protocol 
modules 56A, 56B, . . . 54N, and has an output provided to the video display 12. The diagnostic 
processor can include special purpose hardware, general-purpose hardware with special-purpose 
software, or a combination of the two. The diagnostic processor also includes an input device 
58, which is operatively connected to the general-purpose operating module. A storage device 
60, and a printer are also operatively connected to the general-purpose operating module. 
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In operation, referring to Figs. 3 and 4, a surgeon employing the instrument begins by 
selecting a diagnostic protocol module using the instrument's input device (step 100). Each 
diagnostic protocol module is adapted to detect particular characteristics of the surface of one or 
more types of tissue. For example, the surgeon might select a module which enhances the 
visibility of cancerous tissue. The surgeon would then direct the camera at the area of interest 
and begin inspecting it either under ambient light or with the aid of a supplemental light source, 
which can be filtered to emphasize particular special characteristics of the light it emits. 

The diagnostic processor 38 responds to the surgeon's input by obtaining a series of filter 
transfer functions and an image processing protocol from the selected diagnostic protocol module 
56. The diagnostic processor provides the filtering transfer functions to the LCTF 42 via its filter 
control interface 52 (step 102) and then instructs the image acquisition interface 50 to acquire 
and store the resulting filtered image from the image sensor 46 (step 104). The general -purpose 
operating module 54 repeats these filtering and acquiring steps one or more times, depending on 
the number of filter transfer functions stored in the selected diagnostic protocol module (see step 
106). The filtering transfer functions can represent bandpass, multiple bandpass, or other filter 
characteristics. 

Once the image acquisition interface 50 has stored images for all of the image planes 
specified by the diagnostic protocol chosen by the surgeon, it begins processing these image 
planes based on the image processing protocol from the selected diagnostic protocol module 56N 
(step 108). Processing operations can include general image processing of combined images, 
such as comparing the relative amplitude of the collected light at different wavelengths, adding 
amplitudes of the collected light at different wavelengths, or computing other combinations of 
signals corresponding to the acquired planes. The processing operations can also include more 
complex multivariate statistical techniques to compute the image (e.g., chemometrics). The 
computed image is displayed on the display 12. It can also be stored in the storage device 60 or 
printed out on the printer 62. 

The processing operations can also be based on a diagnostic knowledge base. This 
database can include data resulting from the comparison between optical and actual diagnoses. 
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Each instrument can also continuously update its database as it is used to perform diagnoses, 
thereby steadily expanding its diagnostic capabilities. 

In order to provide a real-time or near-real-time image to the surgeon, the instrument 
repeatedly acquires planes and processes them to develop an image to be displayed to the 
surgeon. This allows the surgeon to move the instrument, or to view moving organs, such as a 
beating heart. This constant acquisition and processing continues until the surgeon either turns 
the instrument off (step 1 10) or selects a different imaging mode (step 1 12). The diagnostic 
processor 38 preferably has sufficient processing power to update the screen in this way at video 
rates (i.e., about 30 frames per second), although rates as low as a few frames per second may 
work quite well, and rates as low as one frame per minute may be adequate for many purposes. 
On slower instruments, general lock-in schemes or other tracking modalities, such as cardiac 
gating, can be used to remove motion artifacts due to breathing or heart beat. Frame rate may 
also be variable, depending on the number of wavelengths and the complexity of the diagnostic 
procedure. 

Preferably, the instrument can operate in multispectral, and hyperspectral, or even 
ultraspectral imaging modes. Multispectral modes involve image processing derived from a 
relatively small number of spectral image planes (two wavelengths to about twenty 
wavelengths). Hyperspectral and ultra spectral imaging modes involve at least twenty image 
planes and can produce significantly more accurate and informative results. Ultraspectral modes 
involve hundreds of wavelengths, and may be able to produce even further information about the 
patient. Hyperspectral and ultraspectral imaging may include selecting specific wavelength 
bands for discrimination of a particular diseased states, or it may also allow the instrument to 
scan for multiple conditions at the same time. 

It is also contemplated that both types of instrument can operate in connection with an 
excitation source, such as an ultraviolet lamp and IR source, or other means of spectral 
illumination or a laser to enhance the received images. Although such excitation may not be 
necessary, it may allow for the examination of different optical phenomenon and provide 
additional diagnostic information. And both emission and reflectance modes can be combined in 
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a diagnostic procedure either simultaneously or sequentially. Relative utilization of different 
emission or reflection measurements involved in the same diagnostic procedure can be obtained 
by modulating the different sources. The instrument can also develop light from bioluminescent 
sources introduced into the patient. 

Instruments according to the invention can also operate to process images from image 
planes acquired at wavelengths outside of the visible region. In one particular embodiment, the 
instrument is sensitive to the visible and near infra-red regions. It is also contemplated that far 
infra-red be included to allow the instrument to sense molecular-specific rotational modes. 

An example of operation would include the use of a diagnostic protocol module that 
examined a first wavelength of about 550 and a second wavelength of about 575 associated with 
oxy- and deoxy-hemoglobin to determine blood oxygenation. The relationship between these 
wavelengths is described in "Hemoglobin: Molecular Genetics and Clinical Aspects," by H. 
Franklin Bunn and Bernard Forget, W. B. Sanders, 1986. Another example would include the 
use of a diagnostic protocol module for examining the Fourier transform infra-red spectra of the 
colon and rectum as described in "Human Colorectal Cancers Display Abnormal Fourier 
Transform Spectra, " by Basil Rigas et aL, Proceedings of the National Academy of Science, pp. 
8140-8144, 1987. 

Surgical and medical applications of instruments according to the invention can include, 
but are not iimited to, determining tissue viability (i.e. whether tissue is dead or living tissue and 
whether it is predicted to remain living), detecting tissue ischemia (e.g., in heart, or in leg after a 
gunshot wound, differentiating between normal and malignant cells and tissues (e.g., delineating 
tumors, dysplasias and precaucerous tissue, detecting metastasis), differentiating between of 
infected and normal (but inflamed) tissue (e.g., extent of aortic root infection), quantification and 
identification of pathogens, (e.g., bacterial count of bum wounds and differentiating and 
delineating other pathologic states. Application can also include tissue, blood chemistry, and 
blood flow (including oxy- and deoxyhemoglobin, myoglobin deoxymyoglobin, cytochrome, 
pH, glucose, calcium and other elements or biological compounds alone or in combination). The 
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instrument can also be applied by veterinarians to animals and by dentists to dental applications, 
such as peridental disease. 

The present invention has now been described in connection with a number of specific 
embodiments thereof. However, numerous modifications which are contemplated as falling 
within the scope of the present invention should now be apparent to those skilled in the art. 
Therefore, it is intended that the scope of the present invention be limited only by the scope of 
the claims appended hereto. In addition, the order of presentation of the claims should not be 
construed to limit the scope of any particular term in the claims. 

What is claimed is: 
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1. A medical instrument, including: 

a first-stage optic responsive to a tissue surface of a patient, 

a spectral separator optically responsive to the first stage optic and having a control input, 
an imaging sensor optically responsive to the spectral separator and having an image data 
output, and 

a diagnostic processor having an image acquisition interface with an input responsive to 
the imaging sensor and a filter control interface having a control output provided to the control 
input of the spectral separator. 

2. The medical instrument of claim 1, wherein the spectral separator is a tunable filter. 

3. The medical instrument of claim 2, wherein the spectral separator is a liquid crystal 
tunable filter. 

4. The medical instrument of claim 1 wherein the imaging sensor is a two-dimensional 
imaging array. 

5. The medical instrument of claim 1 wherein the imaging sensor includes a charge 
coupled device. 

6. The medical instrument of claim 1 wherein the imaging sensor includes an infra-red 
sensitive focal plane array. 

7. The medical instrument of claim 1 wherein the diagnostic processor includes a 
general-purpose processing module and a plurality of diagnostic protocol modules. 

8. The medical instrument of claim 1 wherein the first-stage optic is a macro lens. 

9. The medical instrument of claim 1 wherein the first stage optic is an adjustable lens. 
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10. The medical instrument of claim 9 further including a stand connected relative to the 
first-stage optic to position the first-stage optic relative to the patient. 

1 1 . The medical instrument of claim 1 wherein the first stage optic includes a probe that 
includes an imaging fiber optic cable. 

12. The medical instrument of claim 1 further including a surgical implement attached to 
the probe. 

13. The medical instrument of claim 1 wherein the control interface is operable to adjust 
the filter at least twenty times to acquire hyperspectral data for redisplay in real time 

14. The medical instrument of claim 1 wherein the diagnostic processor includes a 
general-purpose processing module and a plurality of diagnostic protocol modules, wherein each 
of the diagnostic protocol modules includes a plurality of filter transfer functions and an image 
processing protocol, wherein the general-purpose processing module is operative to instruct the 
filter to successively apply the filter transfer functions to light collected from the patient, wherein 
the general-purpose processing module is operative to acquire from the imaging sensor a 
plurality of images of the collected light each obtained after one of the filter transfer functions is 
applied, and wherein the general-purpose processing module is operative to process the acquired 
images according to the image processing protocol to obtain a processed display image. 

15. The medical instrument of claim 14 wherein the general-purpose processor is a real- 
time processor operative to generate a processed display image within a time period on the order 
of the persistence of human vision. 

16. The medical instrument of claim 14 wherein the general-purpose processor is a real- 
time processor operative to generate a processed display image within about one minute. 
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17. The medical instrument of claim 16 wherein the general processor is operative to 
acquire some images more slowly depending on the number of wavelengths and complexity of 
diagnostic protocols. 

18. The medical instrument of claim 1 wherein the diagnostic processor includes a 
general-purpose processing module and a diagnostic protocol module, wherein the diagnostic 
protocol module includes a plurality of filter transfer functions and an image processing protocol 
adapted to detect more than one condition, wherein the general-purpose processing module is 
operative to instruct the filter to successively apply the filter transfer functions to light collected 
from the patient, wherein the general-purpose processing module is operative to acquire from the 
imaging sensor a plurality of images of the collected light each obtained after one of the filter 
transfer functions is applied, and wherein the general-purpose processing module is operative to 
process the acquired images according to the image processing protocol to obtain a processed 
display image. 

19. The medical instrument of claim 1 wherein the diagnostic processor includes a real- 
time processor operative to generate a processed display image within a time period on the order 
of the persistence of human vision. 

20. The medical instrument of claim 1 wherein the diagnostic processor is operable to 
perform diagnostic processing for images acquired from a source that includes visible light. 

21 . The medical instrument of claim 1 wherein the filter and sensor are operable in the 
visible and far infra-red regions. 
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22. The medical instrument of claim 1 wherein the filter and sensor are operable in the ultra- 
violet, visible, and infra-red regions. 

TRADOCS: 1053380.1 (mksk01l.doc) 
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